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Multimedia messaging service and system 



Multimedia Messaging Service (MMS) is a standard that lets users send and 
receive multimedia messages with formatted text, graphics, audio and video clips. MMS 
messages can be sent to (MMS enabled) mobile phones or standard e-mail addresses. The 
sending entity can for instance be a mobile phone or an application running on a computer. 

5 

Fig. 1 shows the generic MMS Person-to-Person flow. A first person wants to 
send an MMS message to a second perpon. He takes his mobile phone, composes the 
message and includes one or more multimedia objects in the message. After hitting the 
"send" button, the multimedia message (text and the multimedia objects) is transmitted over 

10 the mobile network to a Multimedia Messaging Service Center (MMSC). 

The mobile network could be e.g. GSM, GPRS, UMTS or a comparable 
system. As multimedia messages can potentially grow very big, a broadband mobile network 
is preferred. The first person could also have composed the message on Ms personal 
computer connected to the Internet The message would then have been sent to the MMSC 

1 5 over the Internet Other ways to transmit multimedia messages are of course also possible. It 
is important to note that the message is always sent to the MMSC and not directly to the 
recipient. This architecture is an example of a store-and-forward messaging system. 

The multimedia message is received by the MMSC. From there it is forwarded 
to the intended recipient (the second person). The multimedia message is once more 

20 transmitted over amobile network, this time to the moluie telephone of the second person. 
Upon arrival of the multimedia message, this mobile telephone processes the content and 
presents the message to the second person. The two mobile networks in question may or may 
not be the same. 

Jh the above example, the first pexson used the second person's telephone 
25 number as the address to which the multimedia message was to be sent He could also have 
used the second person's e-mail address. The multimedia message would then have been 
forwarded by the MMSC to the second person's e-mail box, from which the second person 
could have read it using a standard e-mail client. 
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2 01.05.2002 
Fig. 2 shows the generic MMS AppUcation-to-Person flow. Multimedia 
messaging is of course not restricted to messages seat by persons to persons. Many 
automated services can be offered, For instance, a popular service is the making available of 
ringtones or logos for mobile telephones. In such an application, a user can send a request for 
5 a particular ringtone or logo to a server from his mobile telephone. An application on the 
server retrieves the appropriate ringtone or logo and transmits it as a multimedia message to 
the MMSC. Once the message arrives at the MMSC, it is handled and forwarded in the same 
way as described above with reference to Fig, 1 . When the mobile phone receives the 
multimedia message, it extracts the ringtone or logo. 

10 Delivering ringtones and logos is already very popular with Hie standard text- 

based Short Messaging Service (SMS). However, MMS has a much broader application. A 
potential use of AppKcation-to-Person MMS is the provision of (high) value content An 
example could be a service which can provide the latest music samples or video clips. The 
user pays an extra amount for that message to cover the content licensing costs* as most of 

15 this content is copyrighted or protected by other intellectual property rights. 

A problem ocours if both presented models are combined: The user who paid 
for the content (application-to-person) can forward this message (and thus the content) to 
another user (person-to-person), only paying for the transmission of the message, but not 
paying any license fees due for forwarding the content 

20 

It is an object of the invention to be able to prevent content from being 
forwarded, or at leqst to be able to restrict such forwarding. 

This object is achieved according to the invention in a method and a system 
which is capable of "tagging 93 the content transmitted in a multimedia message with a 
25 watermark before sending it to the recipient Preferably, such tagging is done in the store- 
and-forward component of the system, in the system as described above this would be the 
MMSC. 

An illustration of a preferred embodiment of the invention is shown in Fig. 3, 
In this case, the MMSC is extended with a watermarking system through which all content 
30 present in multimedia messages must pass. The watermarking system detects the/type of 
content in the multimedia messages (e.g. audio, video, text and so on), chooses an 
appropriate watermarking scheme for the detected type of content, and embeds a marker in 
the content using the chosen watermarking scheme. 
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3 01,05.2002 
The marker does not have to be very big, and does not need to contain a lot of 
information. As will become apparent below, the most important aspect is that it is possible 
later to determine that the content has been watermarked. The content of the watermark is not 
very important, but it must be very difficult or impossible to remove a watermark from the 
5 content This makes robust watermarking techniques the best choice, as they typically have a 
tew payload but are very difficult to remove. 

Of course it is not forbidden to add meaningful content as the payload of the 
watermark. For example, the name of the entity that added the watermark, or an identifier for 
the recipient of the multimedia message could be embedded. 
10 The MMSC is also provided with a watermark detector, as shown in Fig. 4. 

Before adding a watermark* the content of the multimedia message is passed through the 
watermark detector. If a marker can be found, this indicates that the content has passed 
through an MMSC previously. This is proof that the content is being forwarded. As such 
forwarding is to be prevented or at least restricted, the MMSC can now take appropriate 
15 action. 

The simplest action is to simply delete the multimedia message rather than 
forwarding it. However, this may not be acceptable to users. Usually one part of the message 
(such as a picture or ringtone) represents forwarded content. Other parts, such as 
accompanying text messages or pictures, are original content The original content should not 
20 be blocked simply because there is also a forwarded part in the message. The MMSC could 
strip out this forwarded piece of content and pass on the remaining parte of the multimedia 
message. Preferably, any parts that have been stripped out are replaced by a message 
indicating that they have been stripped out and why. 

Alternatively, a license fee required for forwarding the content could be 
25 computed. This license fee can then be charged to the sending user's account, if this 
information is available. This could work well e*g -jvith music content, as music rights 
clearinghouses are available to negotiate a general compensation or license fee. 

The MMSC could also convert the content to a relatively low quality version 
before forwarding the content to the intended recipient This way, the intended recipient 
30 would still get the message and the content, but because of the low quality it would not be 
very valuable. He would then be encouraged to download the original version from the MMS 
application himself. 

If a meaningful payload is present in the watermark, the watermark detector 
can extract this payload and take appropriate action based on the extracted information. 
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For example, if the extracted information identities the initial recipient of the 
content being forwarded (i.e. the person who received it from the MMS application server in 
Pig. 2% the MMSC could compare this identifier against an identifier for the sender of the 
intercepted multimedia message. If these two are the same, it could permit forwarding. This 
5 way, the initial recipient can forward content to his friends, but those ftiends cannot pass the 
message on to anyone else. 

Alternatively, the identifier could be used to track down the initial recipient of 
the content, so that he can he charged a fine, or so that he can be blocked from using the 
MMS application. 

10 If the extracted hrfonmtion identifies the MMSC which added the watermark, 

or the operator of one or more MMSCs, the MMSC could allow forwarding of the message if 
it were the same MMSC as the MMSC which added the watermark. This permits forwarding 
only to and from users of the same MMSC. 

The watermarking system and the watermark detector can be embodied as 

15 separate devices to be installed in the MMSC They can also be embodied as one single 
device. They can be embodied in whole or in part in hardware or in software. 

The entity operating the MMSC is able to optimize content at the MMSC for 
the receiving user agent. This can be the adaptation of the format (for images e.g., GIF to 
JPEG), the size (for video e.g., from CIF to QCIF), the bitrate (for audio e.g., 128kbps to 

20 20kbps) and other parameters. This is shown schematically in Fig. 5. Since this media 

mediation system has access to, and knowledge about the multimedia content of the message, 
the watermark embedding and detection systems are preferably positioned in this system. 
An illustrative embodiment of an MMSC is shown schematically in Pig. 6. 
The fbnctionality of the MMSC can be divided conceptually into a server portion and a relay 

25 portion. Briefly speaking, the Server provides the store-and-foiward architecture and the 
Relay trauscodes and delivers the messages. 

A message is received by the receiver component of the server. The message 
is processed to detect how it should be handled and to whom it should be forwarded. In 
accordance with the invention, the contents of the message is processed by a watermark 

30 detector to detect whether the multimedia content of the message has already been forwarded 
If such detection is positive, the MMSC takes appropriate action, as explained above. 

If forwarding of the message is acceptable, there are several possible paths the 
message can take. Usually the message will be passed oa to the relay. The relay takes care of 
the watermark embedding and transcodes the message to a fonn suitable for handling by the 
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recipient The message is then passed on to the mobile network so that it can be delivered to 
the intended recipient 

The message can also be passed on to an SMTP interface. This interface 
formats the multimedia message In accordance with Internet e-mail standards, and then 
5 forwards the message as an Internet e-mail message to the intended recipient The SMTP 
interface may or may not embed a watermark in the multimedia message before forwarding 
it. If the SMTP interface watermarks the content, further forwarding of the resulting e-mail 
message to another recipient via the MMS network becomes impossible. However, as the 
Internet e-mail network is not very strictly regulated, it will be very difKoult to enforce 
10 forwarding restrictions over the e-mail network itself, 

If the recipient is not on the same mobile network as the sender of the 
multimedia message, the relay cannot forward the message itself to the intended recipient. In 
this case, the server passes the message on to an MMSC which is connected to the correct 
mobile network. It is then the responsibility of this other MMSC to embed the appropriate 
15 watermark before forwarding fixe multimedia message to the intended recipient 

The watermark detector can also be positioned in the relay portion of the 
MMSC. This has the advantage that the detector and embedder could e.g. be realized as a 
single device. However, if a particular multimedia message is then passed on to another 
MMSC which does not have a watermark detector in accordance with the invention, there is a 
20 risk tiiat the content is forwarded multiple times, which is not permitted. 

Watermark embedding can also be implemented as part of the transcoder 
module. Content received by an MMSC from an MMS application will often be of relatively 
high quality- As most mobile devices will not be able to directly handle such high quality 
content (due to e.g. low resolution displays, low quality loudspeakers or headphones)., 
25 transcoding to a format suitable for the mobile device is necessary. 

By making the watermark embedding part of the transcoding process, one can 
make sure that the watermark survives the destructive operations associated with the 
transcoding process. For example, if a 24-bit picture in the TIFF format is converted to an 8- 
bit grayscale GIF image, a watermark that is embedded in the color components of the 
30 original image can be severely distorted. If however, the watermark were embedded after the 
conversion to 8-bit grayscale but before the compression to the GIF format, the final picture 
will be correctly watermarked. 

The functionality of the transcoding module can be extended to more than just 
converting between formats, sizes, bitrates etcetera. For instance, an interface with MMS 
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applications could be provided This way, an MMS application could for instance 
dynamically generate content specifically adapted for one particular mobile device. 

It should be noted that the above-mentioned embodiments illustrate rather than 
limit the invention, and that those skilled in the art will be able to design many alternative 
5 embodiments without departing from the scope of the appended claims. The invention is of 
course applicable to any store-and4brward system, not just the MMS system described 
above. 

It is for instance also possible to implement the watermark detection and 
embedding functionality according to the invention in an Internet or Intranet e-mail server 
10 system. This way e-mail messages cannot be used to indefinitely forward multimedia content 
Analysis of the contents of e-mail attachments is already done by antivirus software, and a 
similar approach could be used to implement watermark detection in attachments. 

La the claims, any reference signs placed between parentheses shall not be 
construed as limiting the claim. The word ''comprising 11 does not exclude the presence of 
15 elements or steps other than those listed in a claim. The word V or "an 11 preceding an 

element does not exclude the presence of a plurality of such elements. The invention can be 
implemented by means of hardware comprising several distinct elements, and by means of a 
suitably programmed computer* 

In the device claim enumerating several means, several of these means can be 

■cay 

20 embodied by one and the same item of hardware. The mere feet that certain measures are 
recited in mutually different dependent claims does not indicate that a combination of these 
measures cannot be used to advantage. 
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CLAIMS: 



1 . A method of controlling retransmission of a content item contained in a 
multimedia message, comprising 

receiving the message containing the content item from a sender together with an identifier of 
an intended recipient of the message, 
5 processing the content item to detect the presence or absence of a watermark therein, 
if the absence of a watermark has been detected, 

- causing a watermark to be embedded in the content item, and 

- allowing retransmission of the message including the watermarked content item to the 
intended recipient, 

10 and otherwise restricting retransmission of the message including the content item to the 
intended recipient. 

2. The method as claimed in claim 1, in which, restricting retransmission 
comprises disallowing retransmission of the message. 

15 

3. The method as claimed in claim 1 , in which restricting retransmission 
comprises allowing retransmission of the message but disallowing retransmission of the 
content item. 

20 4. The method as claimed in claim 1, in which restricting retransmission 

comprises 

- charging a fee to the sender of the message, 

- causing a watermark to be embedded in the content item, and 

- allowing retransmission of the message including the watermarked content item to the 
25 intended recipient 



5, A system arranged for controlling retransmission of a content item contained 

in a multimedia message, comprising 
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receiving means for receiving the message containing the content item from a sender together 
with an identifier of an intended recipient of the message, 

watermark detecting means for processing the content item to detect the presence or absence 
of a watermark therein, and for signaling to conditional tetraasnutting means the presence or 
absence of a watermark, 

the conditional retransmitting means being arranged for, conditional upon receiving a signal 
indicating the absence of a watermark, 

~ activating watermark in means for embedding a watermark in the content item, and 

i- activating retransmitting means for retransmitting the message including the watermarked 

content item to the intended recipient, 
and for otherwise restricting retransmission of the message including the content item to the 
intended recipient 

6. A media transcoding system, arranged for transcoding content items in a 

multimedia message to a format suitable for an intended recipient, comprising the system of 
claim 5. 



7, A computer program product arranged for causing a processor to execute the 

method of claim 1. 
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ABSTRACT: 



A system for and method of controlling retransmission of a content item 
contained in a multimedia message. The method comprises: 

receiving the message containing the content item from a sender together with an identifier of 
an intended recipient of the message, 

processing the content item to detect the presence or absence of a watermark therein, 
if the absence of a watermark has been detected, 

- causing a watermark to be embedded in the content item, and 

- allowing retransmission of the message including the watermarked content item to the 
intended recipient, 

and otherwise restricting retransmission of the message including the content item to the 
intended recipient, 



Fig. 1 
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